Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(e2e/knuu): use latest release and prepare for node knuu tests #4086

Merged
merged 11 commits into from
Dec 11, 2024

Conversation

smuu
Copy link
Member

@smuu smuu commented Dec 6, 2024

Closes #4075

Overview

This PR:

  • Uses the latest release of knuu with fixes
  • Prepares the testnet to be used by celestia-node

@smuu smuu requested a review from a team as a code owner December 6, 2024 11:15
@smuu smuu requested review from cmwaters and ninabarbakadze and removed request for a team December 6, 2024 11:15
@celestia-bot celestia-bot requested a review from a team December 6, 2024 11:16
Copy link
Contributor

coderabbitai bot commented Dec 6, 2024

📝 Walkthrough
📝 Walkthrough

Walkthrough

The pull request includes updates to the go.mod file, primarily focusing on dependency version upgrades. The github.com/celestiaorg/knuu dependency has been updated from v0.16.1 to v0.16.2, alongside several other dependencies. Additionally, modifications were made to various files in the test/e2e directory, enhancing logging capabilities and altering method signatures to include a logger parameter. Changes also include updates to address resolution logic and improvements in temporary directory management.

Changes

File Change Summary
go.mod Updated dependency versions: github.com/celestiaorg/knuu from v0.16.1 to v0.16.2, golang.org/x/crypto from v0.27.0 to v0.29.0, golang.org/x/net from v0.29.0 to v0.31.0, golang.org/x/sync from v0.8.0 to v0.9.0, golang.org/x/sys from v0.25.0 to v0.27.0, golang.org/x/term from v0.24.0 to v0.26.0, golang.org/x/text from v0.18.0 to v0.20.0.
test/e2e/testnet/node.go Added logger field to Node struct, updated methods to use the new logger, modified Init method for temporary directory creation, and changed address resolution logic to use hostname instead of IP.
test/e2e/testnet/setup.go Updated MakeConfig function signature to remove ctx parameter and added peers parameter for setting PersistentPeers.
test/e2e/testnet/testnet.go Added chainID and genesisHash fields to Testnet struct, updated New constructor, modified CreateTxClient for temporary directory handling, and refined logging in WaitToSync.
test/e2e/benchmark/benchmark.go Updated NewBenchmarkTest function to include logger parameter, modified internal logic to pass logger to testnet.New and GetGrafanaInfoFromEnvVar.
test/e2e/benchmark/throughput.go Updated multiple functions to include logger parameter in NewBenchmarkTest calls, modified logging statements to use the logger.
test/e2e/major_upgrade_v2.go Updated MajorUpgradeToV2 function to include logger parameter in testnet.New call and removed ctx from RemoteGRPCEndpoints call.
test/e2e/major_upgrade_v3.go Updated MajorUpgradeToV3 function to include logger in testnet.New call and removed ctx from RemoteGRPCEndpoints call.
test/e2e/minor_version_compatibility.go Updated MinorVersionCompatibility function to include logger in testnet.New call and removed ctx from RemoteGRPCEndpoints call.
test/e2e/simple.go Updated E2ESimple function to include logger in testnet.New call and removed ctx from RemoteGRPCEndpoints call.
test/e2e/testnet/txsimNode.go Modified CreateTxClient function to include logger parameter and updated logging calls to use the standard log package.
test/e2e/testnet/util.go Updated GetGrafanaInfoFromEnvVar function to include logger parameter and changed logging statements to use the new logger format.

Assessment against linked issues

Objective Addressed Explanation
Upgrade knuu version (4075)

Possibly related issues

Possibly related PRs

Suggested labels

WS: Maintenance 🔧, WS: V3 3️⃣

Suggested reviewers

  • staheri14
  • ninabarbakadze
  • evan-forbes
  • rootulp
  • cmwaters

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

‼️ IMPORTANT
Auto-reply has been disabled for this repository in the CodeRabbit settings. The CodeRabbit bot will not respond to your replies unless it is explicitly tagged.

  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR. (Experiment)
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Outside diff range and nitpick comments (1)
test/e2e/testnet/node.go (1)

308-308: Remove unused parameter ctx in RemoteAddressGRPC

The ctx parameter in RemoteAddressGRPC is not used. Consider removing it to simplify the method signature.

Apply this diff to remove the unused parameter:

- func (n Node) RemoteAddressGRPC(ctx context.Context) (string, error) {
+ func (n Node) RemoteAddressGRPC() (string, error) {

- hostName := n.Instance.Network().HostName()
- return fmt.Sprintf("%s:%d", hostName, grpcPort), nil
+ hostName := n.Instance.Network().HostName()
+ return fmt.Sprintf("%s:%d", hostName, grpcPort), nil
}

[warning]

Note: Static analysis tool flagged this issue at line 308.

🧰 Tools
🪛 golangci-lint (1.62.2)

[warning] 308-308: unused-parameter: parameter 'ctx' seems to be unused, consider removing or renaming it as _

(revive)

🪛 GitHub Check: lint / golangci-lint

[failure] 308-308:
unused-parameter: parameter 'ctx' seems to be unused, consider removing or renaming it as _ (revive)

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 6447542 and 4290b00.

⛔ Files ignored due to path filters (1)
  • go.sum is excluded by !**/*.sum
📒 Files selected for processing (5)
  • go.mod (2 hunks)
  • test/e2e/testnet/node.go (4 hunks)
  • test/e2e/testnet/setup.go (1 hunks)
  • test/e2e/testnet/test_helpers.go (1 hunks)
  • test/e2e/testnet/testnet.go (9 hunks)
🧰 Additional context used
🪛 golangci-lint (1.62.2)
test/e2e/testnet/node.go

[warning] 308-308: unused-parameter: parameter 'ctx' seems to be unused, consider removing or renaming it as _

(revive)

test/e2e/testnet/testnet.go

[warning] 402-402: superfluous-else: if block ends with a break statement, so drop this else and outdent its block

(revive)

🪛 GitHub Check: lint / golangci-lint
test/e2e/testnet/node.go

[failure] 308-308:
unused-parameter: parameter 'ctx' seems to be unused, consider removing or renaming it as _ (revive)

test/e2e/testnet/testnet.go

[failure] 402-402:
superfluous-else: if block ends with a break statement, so drop this else and outdent its block (revive)

🔇 Additional comments (24)
test/e2e/testnet/testnet.go (8)

36-37: Addition of chainID and genesisHash fields enhances state management

The new fields chainID and genesisHash in the Testnet struct improve the encapsulation of testnet-specific information.


56-57: Initialize new fields in the constructor

Initializing chainID and genesisHash in the New function ensures that these fields are set up correctly when creating a new Testnet instance.


Line range hint 193-197: Ensure correct parameters in CreateTxClient call

Verify that the updated parameters, particularly t.knuu, align with the function signature of CreateTxClient. This ensures proper initialization of the tx simulation client.


Line range hint 308-313: Initialize peers list correctly

The configuration now collects peer addresses from all other nodes. Ensure that peer.AddressP2P(ctx, true) returns valid and reachable addresses to establish proper network connections.


485-487: Addition of ChainID method

The new ChainID method provides easy access to the chainID of the testnet, enhancing code readability and encapsulation.


489-511: Implement GenesisHash method with proper error checks

The GenesisHash method includes necessary checks for uninitialized knuu and empty node lists, ensuring robust error handling when retrieving the genesis hash.


Line range hint 210-214: Confirm correct ownership and permissions for added folders

When adding the keyring directory to the txsim instance, ensure that the ownership "10001:10001" matches the user inside the container to prevent permission issues.

Run the following script to verify ownership:

✅ Verification successful

Ownership configuration is correctly set to match container user

The verification confirms that the ownership "10001:10001" in the code matches the container configuration. The Dockerfile explicitly sets up the celestia user with UID 10001 for security best practices (UIDs below 10,000 are avoided). This ensures proper permissions for the keyring directory when added to the txsim instance.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check the ownership of the keyring directory in the txsim instance.

# Expected: The owner and group should be 10001.

kubectl exec <txsim_instance_name> -- ls -ld /home/celestia/.celestia-app/keyring-test

Length of output: 181


Script:

#!/bin/bash
# Let's check the Dockerfile or container configuration to verify the user ID
fd Dockerfile | xargs cat

# Also check for any user configuration in deployment files
rg -A 5 "user:" 

Length of output: 6073


150-155: Potential issue with temporary directory cleanup

The use of defer os.RemoveAll(tmpDir) will remove the temporary directory upon function exit. Since txsimKeyringDir is within tmpDir, verify that all necessary files are copied to the txsim instance before the directory is removed to prevent missing files.

Run the following script to check if the keyring directory is correctly copied:

✅ Verification successful

No issue with temporary directory cleanup

The code is safe. The txsimKeyringDir is properly copied to the remote storage before the temporary directory cleanup occurs. This is evident from the code which shows:

  1. The keyring is created and populated with a new mnemonic
  2. The directory is explicitly copied to remote storage via txsim.Instance.Storage().AddFolder(txsimKeyringDir, remoteRootDir, "10001:10001")
  3. Only after these operations is the temporary directory cleaned up by the deferred call
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Ensure that txsimKeyringDir contents are present in the txsim instance storage.

# Expected: The keyring files should exist in the specified remote directory.
# Replace <txsim_instance_name> with the actual instance name.

kubectl exec <txsim_instance_name> -- ls /home/celestia/.celestia-app/keyring-test

Length of output: 177


Script:

#!/bin/bash
# Let's analyze the code around the temporary directory usage to understand how txsimKeyringDir is used

# First, let's find the full context of the function
rg -B 10 -A 20 'tmpDir, err := os\.MkdirTemp\("", "e2e_test_"\)' test/e2e/testnet/testnet.go

# Also check for any usage of txsimKeyringDir
rg "txsimKeyringDir" test/e2e/testnet/testnet.go

Length of output: 1669

test/e2e/testnet/test_helpers.go (6)

13-17: Addition of AssertGreaterOrEqual function

The AssertGreaterOrEqual function enhances test assertions by checking if a value is greater than or equal to another, improving test robustness.


19-23: Addition of AssertLessOrEqual function

The AssertLessOrEqual function allows tests to assert that a value is less than or equal to another, contributing to more precise testing.


25-29: Addition of AssertEqual function

The AssertEqual function simplifies equality checks in tests, making test code cleaner and more consistent.


31-35: Addition of AssertNotEqual function

The AssertNotEqual function provides a straightforward way to assert inequality in tests, enhancing test coverage.


37-41: Addition of AssertGreater function

The AssertGreater function enables assertions that a value is strictly greater than another, which is useful for various test scenarios.


43-47: Addition of AssertLess function

The AssertLess function allows for assertions where a value must be strictly less than another, enhancing the test suite's capabilities.

test/e2e/testnet/setup.go (2)

23-23: Set PersistentPeers correctly

Setting cfg.P2P.PersistentPeers using strings.Join(peers, ",") ensures that the node configuration includes the correct list of peers for network communication.


16-23: ⚠️ Potential issue

Update MakeConfig function to accept peers list

The MakeConfig function signature now includes a peers []string parameter, replacing the previous use of node.InitialPeers. Ensure that all calls to MakeConfig are updated accordingly and that the peers list is correctly populated.

Apply this diff to update the function signature in all calling locations:

-func MakeConfig(ctx context.Context, node *Node, opts ...Option) (*config.Config, error) {
+func MakeConfig(ctx context.Context, node *Node, peers []string, opts ...Option) (*config.Config, error) {

Additionally, verify that peers is properly passed wherever MakeConfig is called.

Likely invalid or redundant comment.

test/e2e/testnet/node.go (6)

202-207: Use of temporary directory for node initialization

Creating a unique temporary directory for each node using os.MkdirTemp enhances isolation and prevents conflicts between nodes.


280-281: Use hostname instead of IP for P2P addresses

Using n.Instance.Network().HostName() for the host part of the address ensures proper resolution within the cluster environment.


303-304: Retrieve gRPC address using hostname

The RemoteAddressGRPC method now constructs the address using the node's hostname, which is appropriate in a Kubernetes context.


309-310: Retrieve RPC address using hostname

Consistently using the hostname in RemoteAddressRPC improves network reliability within the cluster.


318-319: Update tracing address retrieval

The RemoteAddressTracing method now also uses the hostname, ensuring consistency across all network addresses.


221-221: ⚠️ Potential issue

Update MakeConfig function call with peers parameter

The MakeConfig function now requires a peers parameter. Ensure that the peers list is correctly passed to configure the node's persistent peers.

Apply this diff to pass the peers list:

- cfg, err := MakeConfig(ctx, n, configOptions...)
+ cfg, err := MakeConfig(ctx, n, peers, configOptions...)

Likely invalid or redundant comment.

go.mod (2)

11-11: Update knuu dependency to version v0.16.2

Upgrading github.com/celestiaorg/knuu to v0.16.2 ensures the project benefits from the latest features and bug fixes.


230-236: Update golang.org/x dependencies to latest versions

Updating the following dependencies enhances security and compatibility:

  • golang.org/x/crypto to v0.29.0
  • golang.org/x/net to v0.31.0
  • golang.org/x/sync to v0.9.0
  • golang.org/x/sys to v0.27.0
  • golang.org/x/term to v0.26.0
  • golang.org/x/text to v0.20.0

Run the following script to check for any breaking changes or deprecations in the updated packages:

Ensure that the updates do not introduce any incompatibilities with existing code.

✅ Verification successful

Dependency updates appear safe to proceed

The codebase only directly uses golang.org/x/exp packages (exp/slices and exp/constraints), while all the dependencies being updated are marked as indirect dependencies. Since indirect dependencies are not directly imported in the code, and these are minor version updates from the Go team (which maintains backward compatibility), the updates should be safe.

  • Direct usage found in:
    • x/blob/types/payforblob.go: uses golang.org/x/exp/slices
    • pkg/da/data_availability_header.go: uses golang.org/x/exp/constraints
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check release notes for updated golang.org/x packages.

packages=(
  "[email protected]"
  "[email protected]"
  "[email protected]"
  "[email protected]"
  "[email protected]"
  "[email protected]"
)

for pkg in "${packages[@]}"
do
  echo "Checking $pkg for breaking changes..."
  # Open the release notes page for manual review.
  xdg-open "https://pkg.go.dev/golang.org/x/${pkg%%@*}/v0.0.0-${pkg#*@}"
done

Length of output: 1862


Script:

#!/bin/bash
# Let's check how these dependencies are used in the codebase
for pkg in crypto net sync sys term text; do
  echo "=== Checking usage of golang.org/x/$pkg ==="
  rg "\"golang\.org/x/$pkg\"" -A 2
done

# Also check if there are any Go files that might be affected
echo -e "\n=== Checking Go files that import these packages ==="
fd -e go --exec grep -l "golang.org/x/"

Length of output: 1264


Script:

#!/bin/bash
# Let's check the specific imports in the identified files
echo "=== Checking imports in payforblob.go ==="
head -n 20 ./x/blob/types/payforblob.go | grep "golang.org/x/"

echo -e "\n=== Checking imports in data_availability_header.go ==="
head -n 20 ./pkg/da/data_availability_header.go | grep "golang.org/x/"

# Let's also check for indirect dependencies in go.mod
echo -e "\n=== Checking indirect dependencies in go.mod ==="
grep "golang.org/x/" go.mod

Length of output: 918

test/e2e/testnet/testnet.go Outdated Show resolved Hide resolved
rootulp
rootulp previously approved these changes Dec 6, 2024
Copy link
Member

@ninabarbakadze ninabarbakadze left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks sam! lint needs fixing before it can merge

@celestia-bot celestia-bot requested review from a team and rach-id and removed request for a team December 9, 2024 13:15
@celestia-bot celestia-bot requested a review from a team December 9, 2024 13:35
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Outside diff range and nitpick comments (10)
test/e2e/testnet/util.go (1)

Line range hint 14-30: LGTM! Consider enhancing error handling.

The migration from zerolog to the standard logger is well-implemented. However, consider adding debug-level logging for the actual values when environment variables are found (excluding the token for security).

 func GetGrafanaInfoFromEnvVar(logger *log.Logger) *GrafanaInfo {
 	logger.Println("Checking Grafana environment variables")
 	if os.Getenv("GRAFANA_ENDPOINT") == "" ||
 		os.Getenv("GRAFANA_USERNAME") == "" ||
 		os.Getenv("GRAFANA_TOKEN") == "" {
 		logger.Println("No Grafana environment variables found")
 		return nil
 	}
 	logger.Println("Grafana environment variables found")
+	logger.Printf("Grafana endpoint: %s, username: %s\n", 
+		os.Getenv("GRAFANA_ENDPOINT"), 
+		os.Getenv("GRAFANA_USERNAME"))
 	return &GrafanaInfo{
test/e2e/testnet/txsimNode.go (2)

31-31: LGTM! Consider enhancing error logging.

The migration to standard logger is well-implemented. However, the error logging could be more detailed to aid in debugging.

-		logger.Println("failed to set image for tx client", "name", name, "image", image, "error", err)
+		logger.Printf("failed to set image for tx client %s: %v\n", name, err)

Also applies to: 50-54


Line range hint 28-93: Consider breaking down the CreateTxClient function.

The function is handling multiple responsibilities including instance creation, resource configuration, and argument building. Consider breaking it down into smaller, focused functions for better maintainability.

Example structure:

func createInstance(ctx context.Context, knuu *knuu.Knuu, name, image string) (*instance.Instance, error) {
    // Instance creation logic
}

func configureResources(instance *instance.Instance, resources Resources, volumePath string) error {
    // Resource configuration logic
}

func buildTxSimArgs(endpoint string, seed int64, config TxSimConfig) []string {
    // Argument building logic
}
test/e2e/simple.go (1)

Line range hint 1-93: Consider parameterizing test duration and transaction threshold

The hardcoded values for test duration (30 seconds) and minimum transactions (10) could be made configurable to allow for different test scenarios.

Consider adding these as parameters or constants:

 func E2ESimple(logger *log.Logger) error {
     const (
         testName = "E2ESimple"
+        testDuration = 30 * time.Second
+        minExpectedTxs = 10
     )
     // ... rest of the code ...
-    logger.Println("Waiting for 30 seconds to produce blocks")
-    time.Sleep(30 * time.Second)
+    logger.Printf("Waiting for %v to produce blocks", testDuration)
+    time.Sleep(testDuration)
     // ... rest of the code ...
-    if totalTxs < 10 {
-        return fmt.Errorf("expected at least 10 transactions, got %d", totalTxs)
+    if totalTxs < minExpectedTxs {
+        return fmt.Errorf("expected at least %d transactions, got %d", minExpectedTxs, totalTxs)
     }
test/e2e/minor_version_compatibility.go (1)

Line range hint 143-152: Enhance version retrieval robustness

The getAllVersions function could benefit from additional error handling and validation.

Consider this enhanced implementation:

 func getAllVersions() (string, error) {
+    // Ensure we're in a git repository
+    if _, err := exec.Command("git", "rev-parse", "--git-dir").Output(); err != nil {
+        return "", fmt.Errorf("not in a git repository: %v", err)
+    }
+
     cmd := exec.Command("git", "tag", "-l")
     output, err := cmd.Output()
     if err != nil {
         return "", fmt.Errorf("failed to get git tags: %v", err)
     }
-    allVersions := strings.Split(strings.TrimSpace(string(output)), "\n")
+    versions := strings.Split(strings.TrimSpace(string(output)), "\n")
+    if len(versions) == 0 {
+        return "", fmt.Errorf("no git tags found")
+    }
-    return strings.Join(allVersions, " "), nil
+    return strings.Join(versions, " "), nil
 }
test/e2e/major_upgrade_v2.go (1)

Line range hint 122-141: Optimize getHeight function with context timeout

The getHeight function could be simplified by using context timeout directly instead of manual timer management.

Consider this more idiomatic implementation:

 func getHeight(ctx context.Context, client *http.HTTP, period time.Duration) (int64, error) {
-    timer := time.NewTimer(period)
+    ctx, cancel := context.WithTimeout(ctx, period)
+    defer cancel()
+
     ticker := time.NewTicker(100 * time.Millisecond)
+    defer ticker.Stop()
+
     for {
         select {
-        case <-timer.C:
-            return 0, fmt.Errorf("failed to get height after %.2f seconds", period.Seconds())
+        case <-ctx.Done():
+            return 0, fmt.Errorf("failed to get height after %.2f seconds: %v", period.Seconds(), ctx.Err())
         case <-ticker.C:
             status, err := client.Status(ctx)
             if err == nil {
                 return status.SyncInfo.LatestBlockHeight, nil
             }
             if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
                 return 0, err
             }
         }
     }
 }
test/e2e/benchmark/benchmark.go (1)

45-46: Consider enhancing error logging.

While the logger parameter is correctly passed to internal functions, consider using it for error logging instead of returning raw errors.

 testNet, err := testnet.New(logger, kn, testnet.Options{
   Grafana:          testnet.GetGrafanaInfoFromEnvVar(logger),
   ChainID:          manifest.ChainID,
   GenesisModifiers: manifest.GetGenesisModifiers(),
 })
-testnet.NoError("failed to create testnet", err)
+if err != nil {
+  logger.Printf("failed to create testnet: %v", err)
+  return nil, fmt.Errorf("failed to create testnet: %w", err)
+}
test/e2e/testnet/node.go (1)

281-283: Consider keeping context parameter for future extensibility

While using hostname instead of IP is a good change for container environments, removing the context parameter might limit future extensibility (e.g., for timeouts, cancellation, or tracing).

Also applies to: 304-306, 310-312, 319-321

test/e2e/testnet/testnet.go (2)

466-488: Add documentation for lazy loading behavior

The GenesisHash implementation uses lazy loading, which should be documented for better maintainability.

Add a comment explaining the lazy loading behavior:

+// GenesisHash returns the hash of the genesis block.
+// The hash is lazily loaded on the first call and cached for subsequent calls.
 func (t *Testnet) GenesisHash(ctx context.Context) (string, error) {

378-392: Consider making retry parameters configurable

The retry logic uses hardcoded values (20 attempts, linear backoff). Consider making these parameters configurable for different environments.

+const (
+    defaultSyncRetries = 20
+    defaultSyncBackoff = time.Second
+)
+
+type SyncConfig struct {
+    MaxRetries int
+    BackoffDuration time.Duration
+}
🧰 Tools
🪛 GitHub Check: test / test-short

[failure] 387-387:
(*log.Logger).Printf call has arguments but no formatting directives

🪛 GitHub Check: test / test-race

[failure] 387-387:
(*log.Logger).Printf call has arguments but no formatting directives

🪛 GitHub Check: lint / golangci-lint

[failure] 387-387:
printf: (*log.Logger).Printf call has arguments but no formatting directives (govet)

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 34f8d11 and 8f22051.

📒 Files selected for processing (11)
  • test/e2e/benchmark/benchmark.go (2 hunks)
  • test/e2e/benchmark/throughput.go (2 hunks)
  • test/e2e/main.go (1 hunks)
  • test/e2e/major_upgrade_v2.go (1 hunks)
  • test/e2e/major_upgrade_v3.go (1 hunks)
  • test/e2e/minor_version_compatibility.go (1 hunks)
  • test/e2e/simple.go (1 hunks)
  • test/e2e/testnet/node.go (9 hunks)
  • test/e2e/testnet/testnet.go (17 hunks)
  • test/e2e/testnet/txsimNode.go (4 hunks)
  • test/e2e/testnet/util.go (2 hunks)
🧰 Additional context used
🪛 golangci-lint (1.62.2)
test/e2e/main.go

59-59: printf: (*log.Logger).Println call has possible Printf formatting directive %s

(govet)

🪛 GitHub Check: test / test-short
test/e2e/main.go

[failure] 59-59:
(*log.Logger).Println call has possible Printf formatting directive %s

test/e2e/testnet/testnet.go

[failure] 387-387:
(*log.Logger).Printf call has arguments but no formatting directives

🪛 GitHub Check: test / test-race
test/e2e/main.go

[failure] 59-59:
(*log.Logger).Println call has possible Printf formatting directive %s

test/e2e/testnet/testnet.go

[failure] 387-387:
(*log.Logger).Printf call has arguments but no formatting directives

🪛 GitHub Check: test / test-coverage
test/e2e/main.go

[failure] 59-59:
(*log.Logger).Println call has possible Printf formatting directive %s

🪛 GitHub Check: lint / golangci-lint
test/e2e/main.go

[failure] 59-59:
printf: (*log.Logger).Println call has possible Printf formatting directive %s (govet)

test/e2e/testnet/testnet.go

[failure] 387-387:
printf: (*log.Logger).Printf call has arguments but no formatting directives (govet)

🔇 Additional comments (9)
test/e2e/simple.go (1)

33-33: LGTM: Logger parameter addition is consistent with PR objectives

The addition of the logger parameter to testnet.New aligns with the PR's goal of improving test infrastructure.

test/e2e/minor_version_compatibility.go (1)

53-53: LGTM: Logger parameter addition is consistent

The logger parameter addition maintains consistency with other test files.

test/e2e/major_upgrade_v2.go (1)

38-38: LGTM: Logger parameter addition is consistent

The logger parameter addition maintains consistency across test files.

test/e2e/major_upgrade_v3.go (1)

37-37: LGTM: Logger parameter addition aligns with standardization.

The addition of the logger parameter to testnet.New is consistent with the broader effort to standardize logging across the codebase.

test/e2e/benchmark/throughput.go (2)

93-93: LGTM: Consistent logger parameter usage.

The addition of the logger parameter to NewBenchmarkTest call maintains consistency with the updated function signature.


116-117: LGTM: Improved logging using structured logger.

The change properly utilizes the structured logger for ChainID information instead of using the default log package.

test/e2e/benchmark/benchmark.go (1)

27-27: LGTM: Updated function signature for consistent logging.

The addition of the logger parameter to NewBenchmarkTest function signature aligns with the logging standardization effort.

test/e2e/testnet/node.go (2)

58-59: LGTM: Logger integration looks good

The logger field is properly added to the Node struct and correctly initialized in the NewNode function.

Also applies to: 184-184


206-212: LGTM: Proper temporary directory handling

Good implementation of temporary directory creation and cleanup using os.MkdirTemp and defer os.RemoveAll.

test/e2e/main.go Outdated Show resolved Hide resolved
test/e2e/testnet/testnet.go Outdated Show resolved Hide resolved
Signed-off-by: Smuu <[email protected]>
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Outside diff range and nitpick comments (2)
test/e2e/major_upgrade_v2.go (1)

Line range hint 84-93: Consider enhancing version transition logging

The version verification logic is solid, but consider adding more detailed logging around the version transitions to make debugging easier in case of failures.

        resp, err := client.Header(ctx, &heightBefore)
        testnet.NoError("failed to get header", err)
-       logger.Println("Node", i, "is running on version", resp.Header.Version.App)
+       logger.Printf("Node %d version before upgrade (height %d): expected=%d, actual=%d", 
+           i, heightBefore, v1.Version, resp.Header.Version.App)
        if resp.Header.Version.App != v1.Version {
            return fmt.Errorf("version mismatch before upgrade: expected %d, got %d", v1.Version, resp.Header.Version.App)
        }

        resp, err = client.Header(ctx, &upgradeHeight)
        testnet.NoError("failed to get header", err)
+       logger.Printf("Node %d version after upgrade (height %d): expected=%d, actual=%d",
+           i, upgradeHeight, v2.Version, resp.Header.Version.App)
        if resp.Header.Version.App != v2.Version {
            return fmt.Errorf("version mismatch after upgrade: expected %d, got %d", v2.Version, resp.Header.Version.App)
        }
test/e2e/testnet/testnet.go (1)

148-152: Use a more specific pattern for temporary directory.

The current pattern "e2e_test_" could be made more specific to help with debugging and cleanup.

Apply this diff to use a more specific pattern:

-	tmpDir, err := os.MkdirTemp("", "e2e_test_")
+	tmpDir, err := os.MkdirTemp("", fmt.Sprintf("celestia_e2e_test_%s_", name))
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 8f22051 and 29903ae.

📒 Files selected for processing (6)
  • test/e2e/benchmark/benchmark.go (3 hunks)
  • test/e2e/major_upgrade_v2.go (2 hunks)
  • test/e2e/major_upgrade_v3.go (2 hunks)
  • test/e2e/minor_version_compatibility.go (2 hunks)
  • test/e2e/simple.go (2 hunks)
  • test/e2e/testnet/testnet.go (17 hunks)
🚧 Files skipped from review as they are similar to previous changes (4)
  • test/e2e/simple.go
  • test/e2e/minor_version_compatibility.go
  • test/e2e/benchmark/benchmark.go
  • test/e2e/major_upgrade_v3.go
🔇 Additional comments (4)
test/e2e/major_upgrade_v2.go (2)

38-38: LGTM: Logger integration looks good

The addition of the logger parameter to testnet.New() enhances observability while maintaining proper error handling.


63-63: Verify the removal of context from RemoteGRPCEndpoints

The removal of the context parameter from RemoteGRPCEndpoints() might affect timeout and cancellation handling. Let's verify this change is consistent across the codebase.

✅ Verification successful

Based on the search results, I can generate the final response as we have sufficient information. The results show that:

  1. The method definition in test/e2e/testnet/testnet.go shows RemoteGRPCEndpoints() without a context parameter
  2. All calls to this method across the codebase (simple.go, major_upgrade_v3.go, major_upgrade_v2.go, benchmark.go, minor_version_compatibility.go) consistently use the signature without a context parameter

Context removal from RemoteGRPCEndpoints is consistent

The removal of the context parameter appears to be an intentional API change that has been consistently applied across all usages in the test suite. No inconsistencies were found in the method signature or its usage patterns.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Search for all calls to RemoteGRPCEndpoints to ensure consistent usage
rg -A 2 "RemoteGRPCEndpoints\(" test/

# Search for the method definition to verify the intended signature
ast-grep --pattern 'func $_($$) RemoteGRPCEndpoints($$$) {
  $$$
}'

Length of output: 1560

test/e2e/testnet/testnet.go (2)

29-39: LGTM! Well-structured struct modifications.

The new fields enhance the struct's functionality while maintaining good Go practices.


387-387: Fix Printf formatting directive.

The Printf call has arguments but no formatting directives. Use Println instead.

test/e2e/testnet/testnet.go Show resolved Hide resolved
test/e2e/testnet/testnet.go Show resolved Hide resolved
Signed-off-by: Smuu <[email protected]>
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Outside diff range and nitpick comments (2)
test/e2e/testnet/testnet.go (2)

29-39: Consider removing redundant chainID field

The chainID field appears to be redundant since it's already stored in the genesis object. This could lead to inconsistencies if the values get out of sync.

Consider removing the chainID field and using t.genesis.ChainID() instead.


148-153: Consider moving defer after error checks

The defer os.RemoveAll(tmpDir) might execute too early if subsequent operations fail. This could make debugging harder as the temporary files would be removed before they can be inspected.

Consider moving the defer statement after all error checks or passing the temporary directory path to a cleanup function that can be called on success.

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 29903ae and 2cd49ff.

📒 Files selected for processing (1)
  • test/e2e/testnet/testnet.go (17 hunks)
🔇 Additional comments (2)
test/e2e/testnet/testnet.go (2)

466-488: 🛠️ Refactor suggestion

Improve context handling in GenesisHash method

The method caches the genesis hash but doesn't respect context cancellation during the initial fetch.

Apply this diff to improve context handling:

 func (t *Testnet) GenesisHash(ctx context.Context) (string, error) {
+    if ctx.Err() != nil {
+        return "", ctx.Err()
+    }
     if t.genesisHash == "" {
         if t.knuu == nil {
             return "", errors.New("knuu is not initialized")

Likely invalid or redundant comment.


49-61: ⚠️ Potential issue

Add logger validation in constructor

The logger is a required dependency but there's no validation to ensure it's not nil.

Apply this diff to add validation:

 func New(logger *log.Logger, knuu *knuu.Knuu, opts Options) (*Testnet, error) {
+    if logger == nil {
+        return nil, errors.New("logger cannot be nil")
+    }
     opts.setDefaults()
     return &Testnet{

Likely invalid or redundant comment.

test/e2e/testnet/testnet.go Outdated Show resolved Hide resolved
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Outside diff range and nitpick comments (3)
test/e2e/testnet/testnet.go (3)

148-153: Enhance error messages with more context

While the temporary directory handling is robust, the error messages could be more descriptive to aid in debugging.

Apply this diff to improve error messages:

 tmpDir, err := os.MkdirTemp("", "e2e_test_")
 if err != nil {
-    return fmt.Errorf("failed to create temp dir: %w", err)
+    return fmt.Errorf("failed to create temp dir for txsim %s: %w", name, err)
 }

380-396: Consider capping the exponential backoff

The exponential backoff (1<<uint(i)) grows very quickly. By the 10th iteration, the sleep duration would be 1024 seconds.

Apply this diff to cap the backoff:

-time.Sleep(time.Second * time.Duration(1<<uint(i)))
+backoff := time.Second * time.Duration(1<<uint(i))
+maxBackoff := time.Second * 60
+if backoff > maxBackoff {
+    backoff = maxBackoff
+}
+time.Sleep(backoff)

117-120: Enhance log messages with actionable information

While the logging is consistent, some messages could be more actionable by including additional context or suggesting next steps.

Example improvements:

-t.logger.Println("txsim creation failed", "name", name, "grpc_endpoint", grpcEndpoint, "error", err)
+t.logger.Println("txsim creation failed, check endpoint availability", "name", name, "grpc_endpoint", grpcEndpoint, "error", err)

-t.logger.Println("failed to start and create proxy", "name", node.Name, "version", node.Version, "error", err)
+t.logger.Println("failed to start and create proxy, verify network connectivity", "name", node.Name, "version", node.Version, "error", err)

Also applies to: 193-194, 218-221, 424-427

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 2cd49ff and cd0ec72.

📒 Files selected for processing (1)
  • test/e2e/testnet/testnet.go (18 hunks)
🔇 Additional comments (2)
test/e2e/testnet/testnet.go (2)

470-492: 🛠️ Refactor suggestion

Improve context handling in GenesisHash method

The method should respect context cancellation during the initial fetch of the genesis hash.

Apply this diff to improve context handling:

 func (t *Testnet) GenesisHash(ctx context.Context) (string, error) {
+    if ctx.Err() != nil {
+        return "", ctx.Err()
+    }
     if t.genesisHash == "" {
         if t.knuu == nil {
             return "", errors.New("knuu is not initialized")

Likely invalid or redundant comment.


49-61: ⚠️ Potential issue

Add logger validation in constructor

The constructor accepts a logger parameter but doesn't validate that it's not nil. This could lead to nil pointer dereferences later in the code.

Apply this diff to add validation:

 func New(logger *log.Logger, knuu *knuu.Knuu, opts Options) (*Testnet, error) {
+    if logger == nil {
+        return nil, errors.New("logger cannot be nil")
+    }
     opts.setDefaults()
     return &Testnet{

Likely invalid or redundant comment.

Copy link
Member

@evan-forbes evan-forbes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@rootulp rootulp enabled auto-merge (squash) December 11, 2024 14:36
@rootulp rootulp merged commit 76d6c53 into main Dec 11, 2024
29 checks passed
@rootulp rootulp deleted the smuu/20241203-knuu-node-integration-helper branch December 11, 2024 14:45
@rootulp rootulp added the backport:v3.x PR will be backported automatically to the v3.x branch upon merging label Dec 17, 2024
mergify bot pushed a commit that referenced this pull request Dec 17, 2024
…4086)

Closes #4075

## Overview

This PR:
- Uses the latest release of knuu with fixes
- Prepares the testnet to be used by celestia-node

---------

Signed-off-by: Smuu <[email protected]>
(cherry picked from commit 76d6c53)
rootulp pushed a commit that referenced this pull request Dec 18, 2024
…ackport #4086) (#4138)

Closes #4075

## Overview

This PR:
- Uses the latest release of knuu with fixes
- Prepares the testnet to be used by celestia-node
<hr>This is an automatic backport of pull request #4086 done by
[Mergify](https://mergify.com).

Co-authored-by: smuu <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:v3.x PR will be backported automatically to the v3.x branch upon merging external
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Upgrade knuu in e2e
4 participants